Method and apparatus for 3-dimensional vision and inspection of ball and like protrusions of electronic components

ABSTRACT

A method for 3-dimensional vision inspection of objects, such as microelectronic components, which have protrusions as features to be inspected, such as input/output contact balls, is disclosed. The method comprising the steps of determining interior and exterior parameters of a pair of cameras, forming a rectified coordinate system from the parameters of both cameras determined above with a relative pose of second image capturing means with respect to the first image capturing means. A pair of images of the object having a plurality of co-planar protrusions on one surface are captured by the cameras system wherein image coordinates of the images are transformed into coordinates of the rectified coordinate system. Conjugate points of each protrusion are then determined for the measurement of its co-planarity and height. Various configurations of the apparatus for capturing the images and processing the image data are also disclosed.

TECHNICAL FIELD

This invention relates to a method for inspecting objects in3-dimensional vision, including inspecting electronic components whichfeatures' texture may require inspection that is detailed to a sub-pixelscale. An apparatus implementing such method is also disclosed.

BACKGROUND OF THE INVENTION

In a high-precision manufacturing environment where high volume ofprecision small objects are made, such as microelectronics fabricationand packaging of components, it is often necessary for each of thecompleted object such as a microelectronic package to be inspected forquality control purposes. Due to the large volume, such inspection hasto be automated and due to the inspection involving examining specificcritical feature of the package, such as co-planarity of the contactelements in the form of balls in a ball grid array (BGA) device,3-dimensional vision is required.

Some vision systems uses a single axial camera with a plurality ofreflective surfaces to pick up different views of the object to beintegrated in a single image such as that disclosed in U.S. Pat. No.6,055,054 (Beaty, et al.), WO01/04567 and US-2002/37098. Normally, analgorithm in the form of software is used to process the differentreflected views of the object to be integrated into a single image to beinspected.

3-dimensional view in true stereoscopic vision can only be achieved byhaving at least 2 cameras such as that disclosed in U.S. Pat. No.6,778,282 (ICOS Vision Systems), U.S. Pat. No. 6,064,756 and U.S. Pat.No. 6,064,757 (both by Beaty, of al.). These vision systems generallyemploy a first camera to capture an incident view of the object and asecond camera to take a second view from another angle whether directlyor indirectly via reflective surfaces.

Our present method for stereoscopic vision of inspecting objects makesuse of certain functions of Halcon™ Imaging Processing Library (version7.1) by MVTec Software GmbH of München, Germany. Some of Halcon™references are available from its website athttp://www.mvtec.com/download/documentation/documentation.html,including its user manual on 3-dimensional vision system entitledMachine Vision in World Coordinates available athttp://www.mvtec.com/download/documentation/pdf-7.1/3DMachineVision.pdfwhich, along with other relevant Halcon™ publications, are incorporatedherein by reference.

The stereoscopic camera system comprises two cameras trained on the sameobject from different positions. With a pair of cameras, a pair of imagepoints from the projection of a point on the object may be captured. Theimage points are often referred to as “conjugate points” or “homologouspoints”. By making use of interior parameters of both cameras and therelative pose of the second camera in respect of the first cameraobtained during a prior calibration process, the distance of the saidobject point from the stereoscopic camera system can be determined.

A simplified configuration of two parallel-trained single-dimensioncameras with identical interior parameters may be shown in FIG. 1 (PriorArt). The basis, i.e. the straight line connecting the two opticalcentres of the two cameras, may be assumed to be coincident with thex-axis of the first camera. Then, the image plane coordinates of theprojections of the point P (x^(c), z^(c)) may be expressed into twoimages, u₁ and u₂, as follows:

$u_{1} = {f\frac{x_{c}}{z_{c}}}$ $u_{2} = {f\frac{x^{c} - b}{z^{c}}}$

where f is the focal length and b the length of the basis. Thedifference between the two image locations of the conjugate points iscalled the disparity d, which may be calculated as follows:

$d = {\left( {u_{1} - u_{2}} \right) = \frac{f \cdot b}{z^{c}}}$

Given the camera parameters and the image coordinates of two conjugatepoints, the z^(c) coordinate of the corresponding object point P, i.e.the distance from the stereo camera system, can be computed from thefollowing equation:

$z^{c} = \frac{f \cdot b}{d}$

It should be noted that the interior camera parameters of both camerasand the relative pose of the second camera in relation to the firstcamera are necessary to determine the distance of P from the stereocamera system.

Two basic operations that need to be solved for conventional stereovision are:

Step 1—Determination of the interior parameters of both cameras and therelative pose between them during initial set-up, and

Step 2—Determination of a pair of conjugate points for each object pointof interest in stereo image pair captured during inspection to computethe said object points' distances from the stereoscopic camera system.

The first operation involves calibration of the stereoscopic camerasystem whereby a calibration plate is placed such that it liescompletely within both the field of view of the stereo image pair. WithHalcon™ Imaging Processing Library, the calibration of both images maybe carried out simultaneously with the operator termed as “binocularcalibration” [see section 7.3, page 94 of Machine Vision in WorldCoordinates, Edition 2, Halcon™ 7.1.0, published July 2005]. After asuccessful calibration, a virtual stereoscopic camera system calledrectified stereoscopic camera system, of which the rectified coordinatesystem is with respect to 1^(st) camera, will be constructed.

As the accuracy of the 3-dimensional image representation is influencedby the manner in which the cameras are placed, it is important thatneither the interior camera parameters (e.g., the focal length) nor therelative pose (e.g., the distance and orientation between the twocameras) of the two cameras changes during the calibration process orbetween the calibration process, as well as during the ensuingapplication of the calibrated stereo camera system. It is thereforeadvisable to mount the two cameras on a stable platform.

The second operation involves what is called “stereo matching process”,which in Halcon™ processing involves invoking the operators “binoculardisparity” or “binocular distance” [see section 7.4, page 97, op.cit.]which process the necessary calculations to obtain worldcoordinates from the stereo images. In Halcon™ processing, two imagepoints from a stereo image pair captured from the same object point arereferred to as conjugate points. From the principle of stereovision, thedifference between image locations of the conjugate points, which iscalled the “disparity”, can be used to compute the distance of an objectpoint from the rectified stereoscopic camera system.

The Halcon™ Library provides individual functions to compute disparityor distance of the entire overlapping region of the stereo image pair.These functions must first carry out matching operation to establishconjugate points for all image points within the overlapping region ofstereo image pair before disparity or distance profile of the region canbe computed. The stereo image pair must first be rectified, based oninformation gathered from calibration, to align their overall Y imagecoordinate. Thus the ensuing matching process is mainly to find out thedifference of conjugate points in X image coordinate. There are 3different methods, namely “summed absolute difference”, “summed squareddifferences” and “normalized cross correlation”, available for thematching function and all 3 are based on comparison of gray-levelpattern within a small matching window. To obtain accurate results, theobject's surface must have enough textural information to be captured.

Unfortunately, this approach is not useful when determining the distanceto a small feature having varying surface curvature (hence varyingdistance) such as a contact solder ball tip of a ball grid arraysemiconductor package. Using Halcon™ matching methods would becumbersome and inaccurate for the purpose of determining the distance tothe centre point of each ball or ball-like protrusion's curved surfaceas ball surface is void of textural features and is therefore notsuitable for Halcon™ matching methods.

Nevertheless, Halcon™ methodology also provides functions to computedistance from disparity of conjugate points or distance from theconjugate points directly whereby the conjugate points are to bedetermined by the user. It should be noted that conjugate points can betaken as rectified coordinates of original image coordinates.

We have now able to detect the conjugate point of ball tips using ourown methodology and yet be able to compute the distance of every balltip from rectified stereo cameras using Halcon functions. Our processmay be more readily understood from the following disclosure.

SUMMARY OF DISCLOSURE

Our present invention concerns a method for 3-dimensional visioninspection of objects, including microelectronic components havingprotrusions as features to be inspected, wherein the protrusions havesurfaces that are non-planar and/or curvatures such as the tips of pinsor balls. In the general embodiment, our method comprises the steps of:

-   (a) providing at least a pair of image capturing means with similar    interior parameters arranged in a stereovision configuration;-   (b) calibrating each of said image capturing means to determine the    interior and exterior parameters of said image capturing means;-   (c) forming a rectified coordinate system from the parameters of    both image capturing means determined above with a relative pose of    second image capturing means with respect to the first image    capturing means;-   (d) capturing an image of an object to be inspected using each of    said image-capturing means, wherein said object having a plurality    of co-planar protrusions and said image captured is of at least one    surface curvature of each of said protrusion;-   (e) rectifying coordinates of each image to produce a pair of    rectified images in the said rectified coordinate system;-   (f) determining a conjugate point for the centre of each protrusion    on each of the pair of images captured; and-   (g) measuring the distance of each protrusion from at least one of    the image capturing means within the rectified coordinate system and    the co-planarity of the protrusions.

In one preferred embodiment of our method, the aforesaid step (g)measuring the distance of each protrusion from at least one of the imagecapturing means and calculation of co-planarity of the protrusionscomprises determining from the rectified coordinates using establishedmathematical functions the distance of each of the protrusions from atleast one of the image capturing means.

Another preferred embodiment of our method further includes step (h),i.e. calculating the co-planarity of the protrusions based on thedistance of each protrusion to form a seating plane using standardbest-fit plane algorithm. Preferably, our method further still includestep (i) i.e. calculating the height of the protrusions wherein thesubstrate plane is determined with reference to the rectifiedcoordinates; and the perpendicular distance of protrusion to thesubstrate plane is taken as the protrusion height.

In one aspect of our invention, the protrusion height is calculated by(i) approximating with a second plane, in addition to the first seatingplane, that is constructed with protrusions having good co-planarityvalues; (ii) shifting said second plane by the amount of nominalprotrusion height to form the substrate plane; and (iii) taking theheight of the protrusion as the perpendicular distance of saidprotrusion to the substrate plane.

In a preferred embodiment, the calibration in step (b) is determined inrelation to the measurement plane. Preferably, the measurement plane isprovided in the form of a calibration grid.

In another preferred embodiment, the conjugate points of each protrusionbeing determined in step (f) includes (i) locating in each rectifiedimage, the edges of the object to draw up the boundaries of the object;and (ii) performing standard blob analysis to locate the gross positionof each protrusion. Preferably, step (f) further includes (iii)projecting two edge-detecting windows onto each protrusion to locate itstop and bottom edges up to sub-pixel accuracy; (iv) projecting a thirdedge-detecting window from a central vertical position of the protrusionto detect the left edge of the protrusion in right image and the rightedge of the protrusion in the left image up to sub-pixel accuracy; and(v) assigning conjugate points to be right and left edges of theprotrusion.

In yet another embodiment of our method, step (g) further includes (vi)calculating disparity of the protrusion to be the difference between thepositions of the said right and left edges; and (vii) determining thedistance of the protrusion from the stereoscopic camera configurationutilizing the disparity and the relative pose computed duringcalibration.

Preferably, the protrusion to be inspected in our method has at leastone of a round edge or ball tip, including pins of a pin grid array(PGA) package, balls of ball grid array (BGA), and like microelectronicpackages.

In another aspect of our invention, an apparatus is provided for3-dimensional vision inspection of non-planar objects, includingmicroelectronic components, wherein, for calibration and/or measurementpurposes. Our apparatus comprises:

-   (a) at least a pair of image capturing devices with similar interior    parameters arranged in a stereovision configuration;-   (b) a pair of reflective surfaces for reflecting an image of an    object to be inspected to be captured by each of said    image-capturing means; and-   (c) means for mounting said object to be inspected to enable said    object's image to be so reflected and so captured;    -   wherein said object has a plurality of co-planar protrusions on        at least one surface or substrate.

Preferably, the object to be inspected is placed within a ring of lightsource and is illuminated thereby. Preferably still, the light sourcecomprises a plurality of light-emitting diodes (LED) with their lightbeam positioned at about an inclination of 25° towards the object to beinspected. A background light source may preferably be further provided.

As a preferred embodiment, the image capturing device comprises a cameraand a frame grabber, and the frame grabbers are further preferablyconnected to a data processing means via a high-speed data transfer busincluding an IEEE-1394 bus (also known as “Firewire”). Preferably, thedata processing means is a computer capable of running at least aprogram for calibrating, rectifying and processing captured images.

In an alternative embodiment, our apparatus may further including athird image capturing device for another imaging process. Preferably,the third image capturing device is placed in between the pair of firsttwo image capturing devices and is configured for inspecting anothertype of feature of the object, or another type of object.

In another alternative embodiment, a calibration reticle for having itsimage captured for calibration process is provided removably mounted.

The reflective surfaces are preferably positioned at an inclination ofabout 22.5° from parallel axis towards the object to be inspected sothat images of said object are reflected at parallel into the imagecapturing devices.

These and other advantages of our invention may be better understoodwith reference to the accompanying drawings and the detailed descriptionin the following.

LIST OF ACCOMPANYING DRAWINGS

The present invention will now be described in detail with reference tothe accompanying drawings that follows, wherein specific embodiments aredescribed as non-limiting examples or illustrations of the workings ofthe invention, in which:

FIG. 1 (Prior Art) shows a simplified configuration of twoparallel-trained single-dimension cameras with identical interiorparameters employed in a conventional imaging system such as Halcon™;

FIG. 2 shows a basic embodiment of an apparatus in elevation view whichenables the method according to our invention;

FIG. 3 shows an embodiment of an apparatus in schematic representation,which includes a back light source for calibration purposes;

FIG. 4 shows a calibration grid in form of a square block for use incalibration steps of our invention;

FIG. 5 shows one embodiment of an apparatus in schematic representation,for inspection purposes;

FIG. 6 shows a second embodiment of an apparatus in schematicrepresentation, which includes a third camera, a light source andmirrors for 2-dimension/3-dimension lead imaging;

FIG. 7 shows a third embodiment of an apparatus in schematicrepresentation, for inspection purposes;

FIG. 8 shows a fourth embodiment of an apparatus in schematicrepresentation, for inspection purposes; and

FIG. 9 shows a fifth embodiment of an apparatus in schematicrepresentation, for inspection purposes.

DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS

In the general embodiment of our invention's method may be implementedby a configuration or arrangement of a pair of cameras (10, 12) andother elements such as lighting elements including ring light (16) andreflective surfaces or mirrors (18) as shown in FIG. 2 in an elevationview.

Our novel method for 3-dimensional vision inspection covers objects (14)such as micro-electronic components having features to be inspectedwhich may include input-output contact points protruding from thepackage, such as pins and balls of microelectronic packages, includingpin grid array (PGA) packages, ball grid array (BGA) and like packages.

Our method includes the steps of:

-   (a) providing at least a pair of image capturing means with similar    interior parameters arranged in a stereovision configuration;-   (b) calibrating each of said image capturing means to determine the    interior and exterior parameters of said image capturing means;-   (c) forming a rectified coordinate system from the parameters of    both image capturing means determined above with a relative pose of    second image capturing means with respect to the first image    capturing means;-   (d) capturing an image of an object to be inspected using each of    said image-capturing means, wherein said object having a plurality    of co-planar protrusions and said image captured is of at least one    surface curvature of each of said protrusion;-   (e) rectifying coordinates of each image to produce a pair of    rectified images in the said rectified coordinate system;-   (f) determining a conjugate point for the centre of each protrusion    on each of the pair of images captured; and-   (g) measuring the distance of each protrusion from at least one of    the image capturing means within the rectified coordinate system and    the co-planarity of the protrusions.

Theoretically after binocular calibration, the distance of everything infront of the stereoscopic camera system can be determined. Even thoughthe relative pose (exterior parameters) derived from binocularcalibration has a so-called “measurement plane” in it (world coordinateof z=0), it is not important in 3D measurement. Practically, thedistance of all object points falling within focusing range of thestereoscopic camera system can be measured properly. Measurement planeis more relevant for 2D measurement whereby the object is planar andalways physically situated on the measurement plane.

The term “image capturing means” used herein is intended to include boththe camera and frame grabber. The camera may cover still camera or videocamera. The term “at least a pair” covers two or more camera wherein atleast two of the cameras are for stereoscopic vision while the third andsubsequent cameras are optional and may or may not relate to inspectingball-like protrusions from an object to be inspected. “Exteriorparameters” includes the parameters defining the spatial arrangement ofsecond camera with respect to the first camera in world coordinate.

As shown in FIG. 2, the configuration of an apparatus which enables thecalibration and/or measurement step outlined above may comprise of:

-   (a) at least a pair of image capturing devices with similar interior    parameters arranged in a stereovision configuration;-   (b) a pair of reflective surfaces for reflecting an image of an    object to be inspected to be captured by each of said    image-capturing means;-   (c) means for mounting said object to be inspected to enable said    object's image to be so reflected and so captured;    -   wherein said object has a plurality of co-planar protrusions on        at least one surface or substrate.

In one preferred embodiment of our method, step (g) concerning measuringthe distance of each protrusion from at least one of the image capturingmeans and the calculation of co-planarity of the protrusions maycomprise of determining from the rectified coordinates using establishedmathematical functions the distance of each of the protrusions from atleast one of the image capturing means.

Preferably, the method further includes the step of calculating theco-planarity of the protrusions based on the distance of each protrusionto form a seating plane using standard best-fit plane algorithm.

Our method for 3-dimensional vision as described above may preferablyfurther include the calculation of the height of protrusion by thefollowing steps:

-   (i) approximating with a second plane, in addition to the first    seating plane, that is constructed with protrusions having good    co-planarity values;-   (ii) shifting said second plane by the amount of nominal protrusion    height to form the substrate plane; and-   (iii) taking the height of a protrusion as the perpendicular    distance of said protrusion to the substrate plane.

The calibration step (b) above may preferably be determined in relationto a measurement plane which may preferably be provided in the form of acalibration grid.

The determination of conjugate points of each protrusion of step (f) maypreferably include (i) locating in each rectified image, the edges ofthe object to draw up the boundaries of the object; and (ii) performingstandard blob analysis to locate the gross position of each protrusion.Preferably, step (f) further includes (iii) projecting twoedge-detecting windows onto each protrusion to locate its top and bottomedges up to sub-pixel accuracy; (iv) projecting a third edge-detectingwindow from a central vertical position of the protrusion to detect theleft edge of the protrusion in right image and the right edge of theprotrusion in the left image up to sub-pixel accuracy; and

step (f) may preferably further includes (v) assigning conjugate pointsto be the right and left edges of the protrusion.

As another preferred embodiment, step (g) may preferably furtherincludes the steps of (vi) calculating disparity of the protrusion to bethe difference between the positions of the said right and left edges;and (vii) determining the distance of the protrusion from thestereoscopic camera configuration utilizing the disparity and therelative pose computed during calibration.

The measuring of distance of each protrusion of step (g) may preferablyinclude (i) calculating disparity of the protrusion to be the differencebetween the positions of the conjugate points. (ii) determining thedistance of the protrusion from the stereoscopic camera system utilizingthe disparity and the relative pose computed during calibration.

It should be explained here that the term “relative pose” is used inthis specification to mean “exterior parameters” of the image capturingmeans or 3D camera's positions relative to one another as well as to theobject to be examined. The “relative pose” term may also be taken tomean “3D poses” employed in Halcon™ methodology. In Halcon™, 3D posesmeans representation of a rigid transformation with 6 parameters, i.e. 3for rotation and 3 for translation (Rot1, Rot2, Rot3, TransX, TransY,TransZ). The principle of poses is that transformation around even anarbitrary axis can still be represented by a sequence of 3 rotationsaround the X, Y and Z axes of a coordinate system.

As shown in FIG. 2, the object to be inspected is preferably placedwithin and is illuminated by a ring of light source. The light sourcering comprises a plurality of light-emitting diodes (LED) with theirlight beam positioned at about an inclination of 25° towards the objectto be inspected. Preferably, a background light source is furtherprovided.

In one preferred embodiment, as shown in FIG. 3, the cameras areconnected to a data processing means via a high-speed data transfer busincluding an IEEE-1394 bus (also known as “Firewire”). Data processingmeans in the form of a computer capable of running at least a programfor calibrating, rectifying and processing captured images is provided.Reflective surfaces may be provided at an inclination of about 22.5°from parallel axis towards the object to be inspected so that images ofsaid object are reflected at parallel into the image capturing devices.

For the purpose of calibration in steps (b) and/or (c), a “calibrationgrid”, also called a “calibration reticle”, may be provided as aremovably mountable block to have its image captured by the stereoscopiccamera system for the calibration process. The block is preferably arectangular board provided with an array of precision dots within arectangular border, as shown in FIG. 4, may be used. As an exemplarycalibration process, 20 positions of the dots may be captured within thefield of vision of each of the cameras. At least 10 of these positionsmay then be selected for the calibration process.

During the calibration process, the image pairs are read in sequence toobtain the locations of the dots within the border. As the precisespatial relationship among the dots are known, Halcon™ functions may beused to compute the interior camera parameters and the spatialrelationship between the two cameras. With these information, a virtualstereo camera system called “rectified stereo camera system” in whichthe “rectified coordinate system” is determined with respect to thefirst camera, can be established according to the Halcon™ method.

The spatial arrangement of the cameras and lighting means may be variedaccording to the features of the object to be captured and processed.For example, as shown in FIG. 3, the object (14) which image is to becaptured, may be placed within ring light (16) to provide peripheralillumination to the object (14). A backlight (20) may additionally beprovided to provide background illumination.

The images reflected from the object (14) need not be captured directlyby the cameras (10, 12). In fact, as the object size is likely to be asmall microchip as is the common form factor of microelectronic packagestoday, it would be difficult to mount the cameras (10, 12) in parallelto capture the images directly. It would be more practical to have theimages captured by the cameras (10, 12) indirectly, i.e. via reflectivesurfaces or mirrors (22, 24) so that the cameras (10, 12) may be placedapart but still parallel to each other. Preferably, the mirrors (22, 24)are placed at about 22.5° inclination from the parallel axis to theobject (or 67.5° inclination from the plane or field of vision uponwhich the object to be captured is placed).

FIG. 5 shows an alternative arrangement wherein the object to beinspected (14) is still placed within a ring light comprising LEDelements (17). In this arrangement, the cameras are positioned at about45° on each side from the incidental axis of the object (14).

FIG. 6 shows another alternative embodiment wherein the object to beexamined is a leaded microelectronic package (15) wherein is featuredwith leads (15 a) protruding from the sides of the package. The package(15) is shown picked up by a vacuum wand comprising a pick head (30) andpick arm (32). A backlight or reflector (34) may be provided so that thepackage (15) may be illuminated from the background. Bar lights (36) areshown provided to illuminate the sides of the package (15), particularlyat the peripheral parts where the leads protrude from the package body.Reflective surfaces or mirrors (38) may be provided at the same level asthe leads to reflect the images to cameras (10, 12). Preferably, theimages are transmitted through optical elements (40), includingnon-linear optical elements, filters, etc. to enhance the image'scontrast.

It is to be noted that a third camera (11) may optionally be provided atthe middle or between the two cameras (10, 12) to enable the images fromthe package's underside be captured as a whole to be compared orrectified against the separate peripheral images of the leads on eachside of the package (15). As with the pair of cameras (10, 12), thisthird camera (11) may be connected via a high-speed data transfer bussuch as Firewire connection (or IEEE-1394 bus). Data processing means inthe form of a computer (50) running a program for calibrating,rectifying and processing captured images may be optionally provided.

FIG. 7 shows a modification of the arrangement of FIG. 3 wherein thethird camera (11) is similarly provided as an optional configuration.Instead of using high speed date transfer bus and still cameras, videocameras may be used for fast throughput examination of themicroelectronic packages or, other objects to be examined, wherein framegrabbers (42) (or other external devices) may be used in line with eachof the video cameras so that the video images may be digitized forprocessing in the computer (50), as shown in FIGS. 8 and 9.

Certain parallelization processes or parallel processing of images inHalcon™ image functions and operators realized by multi-threadedprogramming and multi-core processing means may be used to speed upprocessing time. Other third party image processing algorithms,including frame grabber interfaces, may also be used integrally withHalcon™ in speeding up certain processes of our novel method describedherein. Such modifications, extension or adaptations are not to beconsidered as departures from the scope of the present invention asdefined in the following claims.

1. A method for 3-dimensional vision inspection of objects, includingmicroelectronic components, having protrusions as features to beinspected, wherein said protrusions having surfaces that are non-planarand/or curvatures, said method comprising the steps of: (a) providing atleast a pair of image capturing means with similar interior parametersarranged in a stereovision configuration; (b) calibrating each of saidimage capturing means to determine the interior and exterior parametersof said image capturing means; (c) forming a rectified coordinate systemfrom the parameters of both image capturing means determined above witha relative pose of second image capturing means with respect to thefirst image capturing means; (d) capturing an image of an object to beinspected using each of said image-capturing means, wherein said objecthaving a plurality of co-planar protrusions and said image captured isof at least one surface curvature of each of said protrusion; (e)rectifying coordinates of each image to produce a pair of rectifiedimages in the said rectified coordinate system; (f) determining aconjugate point for the centre of each protrusion on each of the pair ofimages captured; and (g) measuring the distance of each protrusion fromat least one of the image capturing means within the rectifiedcoordinate system and the co-planarity of the protrusions.
 2. A methodfor 3-dimensional vision according to claim 1 wherein step (g) measuringthe distance of each protrusion from at least one of the image capturingmeans and calculation of co-planarity of the protrusions comprising:determining from the rectified coordinates using establishedmathematical functions the distance of each of the protrusions from atleast one of the image capturing means.
 3. A method for 3-dimensionalvision according to claim 1 further including the step of: (h)calculating the co-planarity of the protrusions based on the distance ofeach protrusion to form a seating plane using standard best-fit planealgorithm.
 4. A method for 3-dimensional vision according to claim 3further including the step of: (i) calculating the height of theprotrusions wherein: the substrate plane is determined with reference tothe rectified coordinates; and perpendicular distance of protrusion tothe substrate plane is taken as the protrusion height.
 5. A method for3-dimensional vision according to claim 4 wherein the protrusion heightis calculated as follows: (i) approximating with a second plane, inaddition to the first seating plane, that is constructed withprotrusions having good co-planarity values; (ii) shifting said secondplane by the amount of nominal protrusion height to form the substrateplane; and (iii) taking the height of the protrusion as theperpendicular distance of said protrusion to the substrate plane.
 6. Amethod for 3-dimensional vision according to claim 1 wherein thecalibration in step (b) is determined in relation to the measurementplane.
 7. A method according to claim 6 wherein the measurement plane isprovided in the form of a calibration grid.
 8. A method for3-dimensional vision according to claim 1 wherein the conjugate pointsof each protrusion being determined in step (f) includes (i) locating ineach rectified image, the edges of the object to draw up the boundariesof the object; and (ii) performing standard blob analysis to locate thegross position of each protrusion.
 9. A method for 3-dimensional visionaccording to claim 1 wherein step (f) further includes (iii) projectingtwo edge-detecting windows onto each protrusion to locate its top andbottom edges up to sub-pixel accuracy; (iv) projecting a thirdedge-detecting window from a central vertical position of the protrusionto detect the left edge of the protrusion in right image and the rightedge of the protrusion in the left image up to sub-pixel accuracy; and(v) assigning conjugate points to be right and left edges of theprotrusion.
 10. A method for 3-dimensional vision according to claim 9wherein step (g) further includes (vi) calculating disparity of theprotrusion to be the difference between the positions of the said rightand left edges; and (vii) determining the distance of the protrusionfrom the stereoscopic camera configuration utilizing the disparity andthe relative pose computed during calibration.
 11. A method for3-dimensional vision according to any one of the preceding claimswherein the protrusion to be inspected has at least one of a round edgeor ball tip, including pins of a pin grid array (PGA) package, balls ofball grid array (BGA), and like microelectronic packages.
 12. Anapparatus for 3-dimensional vision inspection of non-planar objects,including microelectronic components, wherein, for calibration and/ormeasurement purposes, said apparatus comprising: (a) at least a pair ofimage capturing devices with similar interior parameters arranged in astereovision configuration; (b) a pair of reflective surfaces forreflecting an image of an object to be inspected to be captured by eachof said image-capturing means; and (c) means for mounting said object tobe inspected to enable said object's image to be so reflected and socaptured; wherein said object has a plurality of co-planar protrusionson at least one surface or substrate.
 13. An apparatus for 3-dimensionalvision inspection according to claim 12 wherein the object to beinspected is placed within a ring of light source and is illuminatedthereby.
 14. An apparatus for 3-dimensional vision inspection accordingto claim 13 wherein the light source comprises a plurality oflight-emitting diodes (LED) with their light beam positioned at about aninclination of 25° towards the object to be inspected.
 15. An apparatusfor 3-dimensional vision inspection according to claim 14 wherein abackground light source is further provided.
 16. An apparatus for3-dimensional vision inspection according to claim 12 wherein the imagecapturing device comprises a camera and a frame grabber.
 17. Anapparatus for 3-dimensional vision inspection according to claim 16wherein the frame grabbers are connected to a data processing means viaa high-speed data transfer bus including an IEEE-1394 bus (also known as“Firewire”).
 18. An apparatus for 3-dimensional vision inspectionaccording to claim 17 wherein the data processing means is a computercapable of running at least a program for calibrating, rectifying andprocessing captured images.
 19. An apparatus for 3-dimensional visioninspection according to any one of claims 12 to 18 further including athird image capturing device for another imaging process.
 20. Anapparatus for 3-dimensional vision inspection according to claim 19wherein the third image capturing device is placed in between the pairof first two image capturing devices and is configured for inspectinganother type of feature of the object, or another type of object.
 21. Anapparatus for 3-dimensional vision inspection according to claim 12wherein a calibration reticle for having its image captured forcalibration process is provided removably mounted.
 22. An apparatus for3-dimensional vision inspection according to claim 12 wherein thereflective surfaces are positioned at an inclination of about 22.5° fromparallel axis towards the object to be inspected so that images of saidobject are reflected at parallel into the image capturing devices. 23.An object which has been inspected according any one of the methods ofclaims 1-11 or with an apparatus according to any one of claims 12-22.24. An object according to claim 23 which is any one of amicroelectronic pin ball array (PGA) and ball grid array (BGA) packages.25. A facility wherein an apparatus according to any one of claims 12 to22 is installed.